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Abstract 

A general nonequilibrium thermodynamic theory is developed for time- 
dependent Langevin dynamics, starting from the common definition of nonequi- 
librium Gibbs entropy. It is shown that the notations appearing in the First 
and the Second Law of thermodynamics could be consistently applied to 
transient nonequilibrium processes. We find out a general equality repre- 
senting the energy balance relation for the system, and further an explicit 
mathematical interpretation for the extended form of the Second Law is pro- 
posed, which only comes into existence in the time-dependent case. More 
applications to several classic thermodynamic processes are finally included. 

Many different kinds of approach have been put forward in the last several 
decades |[T6l . but in contrast to equilibrium systems, with their elegant theoreti- 
cal framework, the understanding of nonequilibrium steady-state systems is still 
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primitive. In 1998, Oono and Paniconi |[T8l proposed a framework of steady state 
thermodynamics, and distinguished the steadily generated heat which is generated 
even when the system remains in a single steady state and the total heat. They call 
the former the "housekeeping heat", which is equal to the entropy production in 
steady state and may come from the chemical driven force in biochemical systems 
|[T9ll22l . Subtracting the housekeeping heat from the total heat defines the excess 
heat, which reflects the time-dependent variation of the system. Moreover, they 
also put forward a phenomenological extended form of the Second Law: "A pro- 
cess converting work into excess heat is irreversible. And 'reversibility' is modulo 
house-keeping heat, which is produced anyway". 

On the other hand, if one needs to study the nonequilibrium thermodynamics 
from a microscopic or mesoscopic point of view, and wants to be less ambitious and 
get rather satisfactory understanding, effective stochastic models maybe the best 
approach to choose. And in recent years it has been realized that a trajectory per- 
spective of stochastic processes might encode surprisingly more information than 
one might expect from traditional thermodynamic arguments |[T3ll26l . Inspired by 
Oono and Paniconi's framework fTSl . Hatano and Sasa 111 derived the first explicit 
expression for the extended form of the Second Law of Thermodynamics between 
steady states, namely TAS > Qex, where S is the general entropy defined in their 
paper, and Qex is the excess heat. 

Therefore, one can actually use Markov chains and diffusion processes as 
mathematical tools to study nonequilibrium states. It could help us break through 
the shackles of former equilibrium and near-equilibrium statistical mechanics, and 
would accomplish a rather self-contained theory of nonequilibrium thermodynam- 
ics, which will be discussed in the present article. 

Here we consider the dynamics of a Brownian particle in a circuit driven by an 
external force, i.e. 
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where ^(t) represents Gaussian white noise whose intensity is 2jkT according to 
the Einstein's relation. We employ periodic conditions as Kurchan and Hatano- 
Sasa have done in their previous works |[T5l [9l |251. This time-dependent system 
is realized by changing the time-dependent potential V{x, t) and nonconservative 
force /. 

Langevin differential equations which govern a random variable X can also 
be reformulated as "Fokker-Planck" differential equations (Fokker-Planck Equa- 
tions), or "master equations", which govern the probability distribution p{x, t) of 
X{t). Denote the drift coefficient b{x, t) = ^ , it says 

dpjx, t) ^ dj{x,t) 
dt dx ' 

where the current j{x, t) = b{x, t)pt{x) — ^ ^^g^J^ ■ 

We write the steady-state probability distribution function as 7r(x, t) for which 
the right-side of Eq. ([U vanishes for any fixed t, i.e. 

a[5(:.,tMx,t)-f _^ 
dx 

First of all, we accept the opinion that for the Second Law, in particular, a 
proper formulation and interpretation of entropy is more subtle. For stochastic 
processes, the so-called Gibbs entropy has already been widely accepted Il8l [141. 
in statistical physics as well as information theory, hence it becomes our starting 
point. 

Denote the real distribution of X{t) at time t as {p{x,t)}, and we define the 
general entropy at time t as 



S{t) = —k J p{x,t) \ogp{x,t)dx. 
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It is widely known that the entropy change dS could be to distinguished in two 
terms fTTl l20l |2TI : the first, d(.S is the transfer of entropy across the boundaries of 
the system, and the second diS is the entropy produced within the system. 

Here, it is easy to derive that 11211 120]| 

= diS + deS = eprit) - hdr(t), (2) 

dt 

where epr{t) = diS = J -^^^^dx ^ is the entropy production rate at time t, 
and hdr{t) = d^S = y / b{x, t)j{x, t)dx is due to the exchange of heat with the 
exterior, called the heat dissipation rate. 

Consequently, although individual particle has no knowledge of ensemble prop- 
erties, or even whether there is an ensemble, U. Seifert successfully developed a 
theory of entropy production along the trajectory ll25l . which was suggested by the 
so-called fluctuation theorem Il3l[mi2l. 

He defined the sample trajectory entropy along the trajectory at time t as 

S{X{t),t) = -k\ogp{X{t),t), 

recalling that p{x,t) is the real distribution of the Langevin dynamics {X(t)} at 
time t. 

The equation for the motion of S{X(t),t) [25, Eq.9] becomes 

^^^MlH = epr{X{t),t) - hdr{X{t),t), 

where epr{X{t),t) = -^^@^U=x(t) + f^\x=xit)X{t) is the sample en- 
tropy production at time t in which X{t) is of the Stratonovitch type, and hdr{X{t),t) 
7 ^^y*^ \^^x(t)^{'t) is the sample heat dissipation in the medium. 

The rational after these identifications for the change rate of entropy becomes 
clear when averaging over all the trajectories. It gets epr{t) = {epr{X{t),t)), and 
hdr(t) = (hdr(X(t),t)), due to an heuristic equality (X\x, t) = 4^ lEU- 
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The sample heat dissipation hdr{X{t),t) could be regarded as the total heat 
conduction Qtot{X{t),t) with the medium, i.e. 

Qtot{X{t),t) = T ■ hdr{X{t),t) = -fbiX{t),t)X{t), 

and its ensemble average Qtot{t) = T • hdr{t). By convention, we take the sign of 
heat to be positive when it flows from the system to the heat bath. 

The idea of decomposing the total heat into a "housekeeping" part and another 
"excess" part was put forward by Oono and Paniconi |18|, and made explicit in 
Langevin systems by Hatano and Sasa |9|. 

Now we could define the other two kinds of heat: the housekeeping heat and 
excess heat along the trajectory HI ID HI: 

QUX,t) = fcT^^^^^U=^(,)X(t), 

and obviously Qtot{X, t) = Qex{X, t) + Qhk(X, t) = jb{X{t),t)X{t). 

After averaging over all the trajectories, we found out that the housekeeping 
heat is always nonnegative, which implies the nonequilibrium essence of the sys- 
tem: 



^ /X /" /, / X kT d\ogiT(x,t) . ,, 
Qhkit) = J-f{b{x,t)-- ' )jix,t)dx 

= / 7(6(x, t) ^^^fp{x, t)dx > 0. 

J 7 ax 

For equilibrium system, Q^^ reduces to the total heat Qtot^ because in this 
case Qhk = due to b{x,t) = ^ ^^"^g}^'^^ . And in time-independent steady 
state, Qex{t) = 0, hence the housekeeping heat Qhk equals the work done by the 
external driven force, which is all dissipated |[T9ll20ll22l . 

However, the situation is quite different for the time-dependent nonequilibrium 
system, in which the housekeeping heat still comes from the work done by some 



5 



external driven force but where does the excess heat come from?We will show 
that its natual origin is just the change of a thermodynamic quantity called general 
internal energy in the form of heat. 
On the other hand, 

Qex{t) = {Qex{X,t)) 

,^ [' dj{x,t) 

= —kT / — log7r(x,t)aa; 

J ox 

dp{x,t) 
= — g-^ — log7r{x, t)dx, 

due to the Fokker-Planck equation ([Til. 

To derive the energy balance condition and introduce the concept of "general 
internal energy", we shall accept the definition of "the dissipative work" done on 
the system |[T0ll6]|: 

and its ensemble average becomes = {W{X{t),t)) = -kT J p{x,t)^^^dx. 

If the system satisfies the detailed balance conditions for all time, i.e. b{x, t) = 
kj^diogn^x^^ then the traditional concept of internal energy exists and both of the 
excess heat and dissipative work contribute to its change, which is the First Law 
undoubtedly 151. Therefore, we believe that the situation will not be essentially 
different even if detailed balance conditions fails. 

Notice that the integral of the dissipative work subtracting the excess heat does 
not depend on the particular "path" taking through the parameter space, namely, 
only depends upon the initial and final states. Thus, there exactly exists a "general 
internal energy" U {t) = —kT f p{x, t) log 'ir{x, t)dx, whose derivative is just the 
difference of the dissipative work and excess heat, i.e. 

dU{t) 



dt 
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QeAt) + W{t). (3) 



For the equilibrium canonical ensemble, it is just the ordinary internal energy 
according to the Maxwell-Boltzmann's law. Hence here Eq.© is just the general- 
ized First Law of thermodynamics. Also from the trajectory view, we can define the 
internal energy along the trajectory U{X{t),t) = — /cTlog7r(X(t), t). Then 

one has {U{X{t),t)) = U{t) and U{X{t),t) = -Qex{X{t),t) + W{X{t),t), 
which implies the First Law is also satisfied along every trajectory. 

Nevertheless, the situation becomes more complicated in nonequilibrium case, 
since there may exists some external driven force which also pumps energy into 
the system but does not contribute to the change of internal energy ||22]| . 

Denote the work done by the external driven force as Edf{t), and we here try 
to figure out its relationship with other quantities defined previously. 

Now we understand that there exist totally two kinds of external works done 
on the system, one is the dissipative work W{t) and the other Edf{t) from the 
external driven force. They result in the change of general internal energy and the 
heat dissipation respectively. Hence according to energy balance, one has 

W{t) + Edf{t) = + T ■ hdr{t). 

On the other hand, we have already known that 

= -Qe.{t) + W{t) = -Qtotit) + Qhk{t) + W{t), 

and also Qtot{t) = T ■ hdr{t). 

Therefore, it yields Edf{t) = Qhk{t) >, which is only known to be valid in 
steady state HHEa before. 

Now, we turn to the Second Law. Although all the thermodynamic quantities 
in the previous sections could be defined along the sample trajectory, the Clausius 
inequality and many other thermodynamic constrains related to the Second Law 
should be interpreted statistically through ensemble average. 
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Based on the elementary definition of free energy in equilibrium thermody- 
namics F = U — TS, here we could define a general free energy in the same 
way: 



F{t) = U{t) - TS{t) = kT j p{x,t)\og 



pix,t) 



tt{x, t) 

For equilibrium system, it is just the Gibbs free energy in a spontaneously 
occurring chemical reaction at constant pressure p and temperature T, and also the 
Helmholtz free energy for systems at constant V and T 1231 . Its change gives the 
maximum work, other than pV work. Therefore, it is called a "hybrid free energy" 
by Ross, J. 1231 . 

From a mathematical point of view, it is just the relative entropy of the distri- 
bution {p{x, t)} with respect to another one {7r(x, t)}. Hong Qian 1211 has proved 
that this relative entropy from information theory could be identified as the free 
energy difference associated with a fluctuating density in equilibrium, and is also 
associated with the distribution deviate from the equilibrium sate in nonequilibrium 
relaxation. 

Then, 

dF{t) _ dUjt) ^ dS{t) 
dt dt dt 

= W{t) - QeAt) - T{epr{t) + hdr{t)) 

= W{t)-{T-epr{t)-Qhk{t)), (4) 

Here we introduce a new concept named Free heat Qf{t) = T- epr (t) — Q^^ {t) 
identifying the free energy change in the form of heat, i.e. 

^ = W{t)-Qf{t), 

which will play the central role in the extended form of the Second Law of Ther- 
modynamics. 

Notice that the entropy production rate epr{t) = diS = J ^^^(2) dx is non- 
negative, hence according to Eqs. ^ and (01), we derived several thermodynamic 
inequalities in the differential form: 
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j. dS{t) 



dt 



+ Qtot{t) 



T ■ epr{t) > 0, 



(5a) 



dF{t) 



W{t)-Qhk{t) 



T ■ epr{t) < 0. 



(5b) 



dt 



Eq. (l5al) is just the well-known Clausius inequality {dS > — ^|?^), which is 
rectified to obtain expressions for the entropy produced (dS) as the result of heat 
exchanges (Qtot)- And Eq. (l5bl ) is a general version of the free energy inequal- 
ity for the amount of work performed on the system, since the work values must 
then be consistent with the Kelvin-Planck statement lU and forbids the systematic 
conversion of heat to work. 

More precise, the quantity Qhk{t) in Eq. (l5bl) vanishes when the detailed bal- 
ance condition holds and then it returns back to the traditional Helmhotz or Gibbs 
free energy inequalities of equilibrium thermodynamics depending on whether it 
is a NVT or NPT system [IJ. In this case, —dF > —W, which implies the de- 
crease of free energy gives the maximum dissipative work done upon the external 
environment. 

Also, their corresponding integral forms are 



Beyond the traditional form of the Second law based on entropy production, we 
would give a rather different but much more general expression that what Hatano 
and Sasa got 191, only need to notice that the free heat 



Then according to Eqs. Q and (01), another group of inequalities in the differ- 
ential form arises: 




(6b) 



(6a) 
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dt 



+ Qex{t) 



T-epr{t)-Qhk{t)>0, 



(7a) 



dF{t) 



W{t) 



Qfit) 



T-epr{t)+Qhk{t) <0. 



(7b) 



dt 



followed by their corresponding integral forms 



TAS + j Qex{t)dt > 0, 
AF- [ W{t)dt < 0. 




(8b) 



(8a) 



Eq. (|8al ) is the extended form of Clausius inequality during any nonequilibrium 
time-dependent process, whose special case is included in Hatano and Sasa's work 
||9l. And Eq. (l8b] ) is a different general form of free energy inequality. It implies 
the dissipative work value must be consistent with the Oono-Paniconi statement of 
the extended Second Law of thermodynamics |[T8l . which forbids the systematic 
conversion of excess heat to work. 

For equilibrium case, Q^x = Qtot, then they both return back to Eq. (l5al ) 
actually. And then if in nonequilibrium steady state, then Qf{t) = 0, and this 
extended form of the Second Law is concealed. 

Finally, we apply the previous formula to several important thermodynamic 
processes: 

First, the relaxation process towards steady states. The relaxation process to- 
wards steady states has been extensively discussed by Glansdorff and Prigogine 
||7l[T71, and then by Schnakenberg for the master-equation systems ll24l . 

Here, the First Law of Thermodynamics becomes 



dU{t) 
dt 



Qex{t)i 



and the energy balance reads 



dU{t) 
dt 



+ T-hdr{t) = Edf(t). 
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Combined with Edf(t) = Qhk{t), it is found that the whole heat dissipation 
is from two sources: one is the excess heat contributing to the change of general 
internal energy; the other is the housekeeping heat caused by the external driven 
force. 

Furthermore, the extended form of the Second Law now gives 

= -T ■ epr{t) + Qhk{t) = -Qf{t) < 0, 

thus F{t) serves as a Lyapunov function for this relaxation process, which actually 
has a solid thermodynamic basis. 

Second, the thermodynamic cyclic process. In equilibrium thermodynamics, 
a thermodynamic cycle is a series of thermodynamic processes which returns a 
system to its initial state. As a conclusion of cyclic process, all the state variables 
should have the same value as they had at the beginning. Thus AU = AS = 
AF = 0. 

But variables such as heat and work are not zero over a cycle, but rather are 
process dependent. The First Law of Thermodynamics dictates that the net heat 
input is equal to the net work output over any cycle, i.e. f W{t)dt = f Qex{t)dt. 

Hence in this case, the former form of the Second Law Q gives 

j Qtot{t)dt = T ■ j epr{t)dt > 0, (9a) 

J Wit)dt > - j Qhkdt. (9b) 

If one rewrite Eq. ^ as j{W{t) + Edf{t))dt = J Qtot{t)dt > 0, then it 
is just the familiar statement of traditional Second Law of Thermodynamics "the 
conversion from work to total heat is irreversible". 

Moreover, the extended form ([8]) gives 

jQex{t)dt = j Qf{t)dt > 0, (10a) 
J W{t)dt = j Qex{t)dt > 0, (10b) 
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which explicitly confirms the claim that "the conversion from work to excess heat 
is irreversible" |fl8l|. In other words, during a cyclic process, not only the total heat 
but also the excess heat could only be from the system into the heat bath rather than 
follow the opposite direction. 

Third, the transition process between two steady states. The previous steady- 
state thermodynamics of Langevin systems is based on a generalized version of the 
Jarzynski work relation ifTTl [T2l [T3l . and concluded that Q^x should correspond 
to the change of a generalized entropy S in an appropriate limit [9|. Actually, 
the extended form of the Second Law derived by Hatano and Sasa |9| is just a 
straightforward consequence of ([8]), which satisfied for any arbitrary transient state. 

It is indispensable to point out that the general entropy defined in their work 
im is just the general internal energy related to the First Law of Thermodynamics 
rather than the general Gibbs entropy in the present article and also in Seifert's 
recent work lf25l . Note that the two quantities are always different except for steady 
states, hence our formula is consistent with that of Hatano and Sasa. 

Summarily, for stochastic systems, the central problem is around the extension 
of the Second Law, which originally describes the fundamental limitation on pos- 
sible transitions between equilibrium states. And recently, it has been studied from 
the trajectory point of view, which stimulated the rapid emergence of so-called 
fluctuation theorems ll26l [T3l . Thus the main purpose of the present article is to 
investigate the relationship and rationality of the thermodynamic functionals along 
the trajectory, especially the present formula here. It would be interesting to test 
experimentally all the quantities and relations, especially in nonharmonic time- 
dependent potentials, where one does not expect Gaussian distributions. Although 
we only study the stochastic process, the thermodynamic relations derived here 
may be satisfied much more generally, especially the extended form of the Second 
Law, since both of the entropy production rate and housekeeping heat could be 
gained phenomenological ifTSll . 
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